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Concepts and their interrelationships shape our perceptions of the world. 
Yet concept development is notoriously difficult to access. This investigation 
used concept mapping, a technique of graphically representing concepts and their 
relationships along two dimensions, to assess changes in structural knowledge of 
prospective teachers at three levels of a teacher education program. 

Scores were calculated for student-constructed pre and post maps, for the 
following: item scores (indicating dif f erentration) , level scores (indicating 
degree of hierarchical organization), and similarity to group, similarity to 
instructor and item stream similarity to instructor scores (indicating development 
of a shared technical vocabulary). Content analysis addressed questions of how 
content of the maps changed within and across courses, and these were compared 
with instructors' intentions for students' thinking. 

Findings from both the quantitative and qualitative analysis indicated stu- 
dents' maps became more differentiated, organized, and evidenced the development 
of a shared technical vocabulary. The types of information concept maps can 
provide are illustrated in this paper. 



CONCEPT MAPPING IN ASSESSING 
Concepts and their interrelationships shape our perceptions of the world, 
and help determine our actions* As teacher educators, one of our goals is to 
build powerful conceptual structures related to the processes involved in 
teaching. Yet conceptual growth is notoriously difficult to assess. Concept 
mapping, a technique of graphically representing concepts and their hierarch- 
ical interrelationships along two dimensions, is a useful technique for exam- 
ining changes in content and organization of individuals' conceptual 
structures. Figure 1 illustrates an example of a concept map constructed by a 
student in an introductory education course. Concept mapping requires stu- 
dents to structure concepts and their interrelationships from memory, and thus 
from their perspective rather than a framework imposed by the instructor or 
test-maker. Much of the recent research on teachers' thinking has examined 
implicit beliefs or theories, inferring them from ethnographic observations, 
or clinical interviews (Clark & Yingsr, 1979). A more direct approach to 
exploring preteachers' concepts related to teaching is described here. 

Theoretical Background 
Lewin (1951), Werner (1978), and Ausabel (1978) suggested that concept 
development involve.^ increased differentiation aniong concepts and increased 
hierarchical organization and integration of concepts. Building on Ausubel's 
(1978) distinction between rote and meaningful learning, Novak (1977) defined 
rote learning as new information "arbitrarily stored in a cog<iitive structure 
or . . • associated with weakly differentiated concepts at low levels of 
abstractness and inclusiveness" (p. 113). Meaningful learning occurs when new 
information is subsumed into a large structure which relates it to other 
concepts. Concept mapping portrays a graphic representation of how a student 
is thinking about a particular area, providing a frflunework to which an 
instructor can link subsequent instruction. 



Concept Mapping in Assessing 
To be useful in assessing structural changes in students* thinking, maps 
should have features which demonstrate these changes that theorists predict. 
One would also expect changes in the content on concept maps resulting from 
instruction. Within a course or program of study, convergence towards a group 
consensus, or towards an instructor's concept map, might be expected. Novak 
(1977) supports the use of concept mapping in assessing conceptual growth. He 
points out that standard achievement measures are inadequate in telling about 
•the specific conceptual hierarchies an individual possesses" (p. 114). Con- 
cept mapping has been widely recommended as a technique for assessing stu- 
dents' structural knowledge (e.g. Bogden, 1977; Moreira, 1979; Novak & Gowin, 
1984), yet few studies have examined quantitative and qualitative differences 
in concept maps produced by individuals with varying degrees of educational 
experience, or before and after instruction. The remainder of this paper 
discusses a recent study on concept mapping and suggests how the procedure 
might be used in assessment, instruction, and program development. 

Study of Preteachers' Concept Maps of Teacher Planning 
Concept mapping is a process by which individuals illustrate their re- 
spective understandings of a content area by letting their thoughts flow 
freely and then hierarchically organizing, in chart form, these free associa- 
tions around the concept, and indicating interrelationships between ideas with 
connectina lines. Superordinate — subordinate relationships, interrelation- 
ships among subordinate concepts, as well as content diversity are graphically 
presented. The following quantitative scores were derived from pre and post 
concept maps of students in three consecutive courses in Syracuse University's 
teacher education program, for the concept teacher planning ; 
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1. item score (indicating degree of differentiation), the number of 
discrete entries included on an individual's concept map. The item score in 
Figure 1 is 59* 

2. item stream score , the nufr->er of distinct superordinate concepts 
attached to the central concept. The item stream score in Figure 1 is 5. 

3. level score (indicating degree of hierarchical organization), the 
largest number of subordinate levels employed in a single item stream. The 
level score in Figure 1 is 6. 

4. similarity to group score (indicating group consensus), the sum of 
tne frequency of each entry in the total group of students at the same educa- 
tional level, divided by the total number of items for that individual. 

5. similarity to instructor score , the number of entries identical to 
entries on the instructor's map. 

^* item stream similarity to instructor score , the number of item 
streams identical to item streams on the instructor's map. 

Pre and post measures, consisting of concept maps constructed at the 
beginning and end of a semester course as part of the course requirement, were 
derived in each of the above areas for three groups of students and their 
instructors, at three levels of a teacher education program. Forty-nine 
students in either a) an introductory education course, called Study of 
Teaching , b) an intermediate course. Personalizing Teaching and Learning , or 
c) an advanced undergraduate course. Strategies of Teaching , participated in 
the study. Scores derived from maps of these students were compared across 
the three groups for a global concept (teacher planning) relating to all three 
courses, and within ee^ch group from the beginning to the end of the semester. 
Quantitative analysis of the maps addressed the question of whether scores 
followed the changes theorists predicted, e.g. increased differentiation, 
Q hierarchical organization, and similarity to instructor* 
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Concept Mapping in Assessing 
Content analysis of the maps addressed the issue of whether there were 
consistent differences in individuals at different l3vels of the program, and 
explored the nature of those differences. The question of whether there were 
changes in maps resulting from direct instruction in a particular content area 
was addressed. Each instructor was interviewed several times across the 
semester regarding their goals for students" thinking regarding teacher plan- 
ning > They were asked to construct maps of how they would like students to 
think about the topic at the end of their course. 

Content analysis dealt with questions such as, *What were the particular 
content concerns of students at the three levels of the program?*; "Were there 
structural differences in maps of students at different levels of the pro-- 
greun?* ; "How did the content of the maps relate to instructors intentions for 
student thinking?*; and, "How did content of students" maps compare with 
content concerns of experienced '^eachers regarding teacher planning?" • The 
qualitative analysis provided information about how a student was thinking 
about a subject area, information which could then be used by an instructor to 
modify instruction. The remaining portion of this paper presents select 
findings from this study illustrating the types of information concept mapping 
can provide. 

Findings and Potential Uses of Concept Mapping 
Correlational data 

Year in school and prior number of education courses were both signif- 
icantly and negatively correlated with students" similarity to instructor 
scores on the pre maps, and significantly and positively correlated with 
students post test similarity to instructor stream scores, as indicated by 
Pearson Product Moment correlations (p<01). That is, more advanced students 
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adopted the instructor's overall organizing concepts more so than less ad- 
vanced students. Interviews with instructors of the courses indicated that 
this was at least partially consistent with their intentions. The instructor 
of the introductory course was primarily concerned with increasing students' 
repertoire of concepts related to teacher planning, whereas the instructor in 
the intermediate course was concerned with having students think about plan- 
ing with a particular (Lewinian) framework. In the advanced course students 
were expected to apply the framework in actual planning for instruction. 

This was one of the many examples of how the qualitative data enhanced 
the interpretability of the quantitative data. This trend was also observed 
in con^aring two subgroups in the advanced course — those who had taken the two 
prerequisite courses vs. those who had not. A paired comparison t-test by 
subgroup indicated that item score gains from pre to post mapping were not 
significant for students who had had the prerequisite courses, but were highly 
significant for students who had not (pfX)001). On the other hand, pre to 
post similarity to i nstructor score gains were highly significant in the group 
who had had the prior course work (pc.C004). Students who had the prior 
coursework brought with them a larger repertoire of concepts related to 
teacher planning (they did have larger mean item scores on the pretests) and 
their learning involved focusing in on those concepts for which they received 
instruction, whereas students who had not had prior coursework were developing 
a broader repertoire of concepts related to teacher planning. 
Pre to post differences in scores derived from concept maps 

Overall, item scores and similarity to instructor scores increased sig- 
nificantly (p. 01) from pre to post mapping as indicated by a General Linear 
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Models Procedure. Similarity to instructor stream scores and similarity to 
arou£ scores also generally increased from pre to post mapping. That is, 
students had increased concept differentiation, and increased shared vocab- 
ulary with each other, and the instructor, after instruction. The expected 
differences between groups at three levels of the program for these scores 
failed to occ r, perhaps due to the small number of students taking the 
courses in the prescribed sequence. Between group differences in map content 
were observed. Content analysis of the maps indicated that more advanced 
students had clearer, more organized maps than beginning students. 
Qualitative changes in maps from pre to post mapping 

Qualitative changes in the maps from pre to post mapping came from sev- 
eral sources: 

a) the researcher analyzed pre and post maps within each group in terms 
of changes in content, organizational structure, clarity of superord- 
inate'-subordinate transitions, and abstractness and inclusiveness of 
concepts. 

b) instructors commented on changes they saw in the maps. 

c) in the introductory course, students were given the opportunity to 
compare their pre and post maps in class and discuss changes they 
observed. These were recorded by the researcher. 

d) in the advanced course, students wrote essays comparing their pre and 
post map and the researcher summarized common themes in these. 

The following changes were common to a large number of maps, and were 
corroborated by the above data sources. As compared to pre maps, post maps 
were more developed in terms of number of items and clarity of overall organ- 
ization, were organized around more abstract, inclusive terms, reflected more 
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course content, and focused more directly on the processes of planning and 
instruction. Many students noted dramatic changes in their thinking from pre 
to post mapping. Most students reacted very positively to having the oppor- 
tunities to organize their thoughts via the mapping, and compare changes in 
their thinking. A few students were uncomfortable with mapping on the pre 
maps, indicating they weren't sure of "the right way" to do it. Most were 
more comfortable the second time around. The following comments from two 
students were typical; 

It was nice doing this assignment so that I am able to see 
the change in my thinking. 

I was shocked to see the differences between my first concept 
map and my current concept map. I could see how much I have learned 
this semester and how my perceptions of teaching had changed, for 
the better. There were many differences in my maps, which I found 
hard to believe. I was surprised to see such a dramatic change in 
my thinking. For instance, the way in which I categorized was 
much more sophisticated, indicating a higher level of thinking. 
Concept mapping seems to have excellent potential as a tool for self-evalua- 
tion of learning gains. 
Content analysis of concept maps 

In order to compare item streams (superordinate concepts) and items 
listed on maps within and across groups, each student's item stream and item 
labels were listed and frequency counts within and across groups were tabu- 
lated for both pre and post maps. Generally, there was great diversity in the 
content and organization of the maps. Three categories common in all three 
groups are tne streams — time , subject , and materials . The categories 
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materials and methods were common for all three groups' post maps. If one 
looks to the literature on teacher planning and effectiveness, one finds that 
these factors are critical both in the way experienced teachers think 
(Crocker, 1983) and in their effectiveness with students (McDonald, 1977). 
There were more commonalities in both item streams and items as the level of 
the course increased, and within each course from pr^' to post mapping ses- 
sions. That is, group consensus seemed to develop within each course and 
within the program as a whole. 

These are but the most general of the findings from the content analysis. 
Such analysis seems highly useful in providing instructors within a course, 
information on which concepts were being assimilated as a result of instruc- 
tion and how these concepts are being organized. Such analysis is also useful 
in program development, providing a profile of students' thinking in each 
course, which can then be compared to the intent of the program, and us^d to 
improve instruction. 



The partial findings of a study on concept mapping of students at three 
levels of a teacher education program were presented to illustrate the types 
of information and analyses that can be provided by concept maps. The tech- 
nique seens congruent with an increased focus by researchers on teachers' 
thinking. Napping fo::uses on how a student is thinking regarding a topic 
area, structured from his or her own perspective. Concept maps have some 
limitations in this respect, as one cannot determine the meaning held by a 
student for a particular term, or why a student sees two concepts as related. 
However, maps do provide an overview of how students are thinking about a 
particular topic. When more depth is desired, maps can serve as the focus for 
an interview with individual students. 



Summary 
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Aside from their documented usefulness in planning for instruction 
(Stewart, VanKirk, and Rowell, 1979), in improving recall of text (Armbruster 
and Anderson, 1980), as a study strategy (Reigeluth, 1979), and in "learning 
to learn" (Novak & Gowin, 1984) maps nre also useful in assessing changes in 
students' structural knowledge. Novak and Gowin (1984) say that the greatest 
significance of concept maps may be in the improvement of evaluation in educa- 
tion. 

Measures derived from concept maps have construct validity in that they 
are derived from Ausubel's theory. However, many learning gains were not 
reflected in quantitative scores derived froir the maps (e.g. increased clarity 
in organization), and the qualitative analysis of the maps was the richest 
source of information about content and organization of students' thinking. 
This information is useful in assessment of student's learning gains, in 
program development, and in students' self -evaluation. Mapping provides de- 
tailed information on the types of information students are acquiring and on 
how they are organizing the information. As with any methodology, there are 
inherent limitations in the concept mapping approach. Problems with using 
scores derived from concept maps as an assessment device include many of the 
same problems as any other free recall measure. For example, irrelevant items 
and repeated items may contribute to inflated item scores. Distortions of 
content or ambiguous responses are not reflected in scores. Several of the 
scores derived from the maps are not independent. If the central topic is too 
broad, it may elicit only a small percentage of the students' repertoire of 
technical vocabulary. Regarding the qualitative analysis of the maps, infer- 
ences, and judgements are made concerning the maps' clarity in organization, 
abstractness, and inclusiveness, which may not be replicable. There are 
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connections and hierarchies on student maps which don*t make sense to an 
outside observer. The focus of the analysis shapes the results. For example, 
in this study, I looked at similarities in maps by analyzing most frequent 
item streams and items from pre to post mapping across the three groups. I 
did not look at changes in least frequent item streams and items, though this 
might have yielded some interesting results. Finally, the relationship be- 
tween students' concept maps, their actual planning and interactive decision- 
making, and their effectiveness as teachers needs to be explored. 
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Fiaure 1 Post Map of Student in the Introductory Course for the concept, 
"teacher planning". 
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